The Evoked Potential Operant Conditioning System (EPOCS): a research tool and an emerging therapy for chronic neuromuscular disorders

The Evoked Potential Operant Conditioning System (EPOCS) is a software tool that implements protocols for operantly conditioning stimulus-triggered muscle responses in people with neuromuscular disorders, which in turn can improve sensorimotor function when applied appropriately. EPOCS monitors the state of specific target muscles (e.g. from surface EMG while standing, or by gait cycle measurements while walking on a treadmill), and automatically triggers calibrated stimulation when pre-defined conditions are met. It provides two forms of feedback that enable a person to learn to increase or decrease the targeted pathway’s excitability. First, it continuously monitors ongoing EMG activity in the target muscle, guiding the person to produce a consistent level of activity suitable for conditioning. Second, it provides immediate feedback of the response size following each stimulation, and indicates whether it reached a target value. To illustrate EPOCS use, this article describes a protocol through which a person can learn to decrease the size of the Hoffmann reflex—the electrically-elicited analog of the spinal stretch reflex—in the soleus muscle. Down-conditioning this pathway’s excitability can improve walking in people with spastic gait due to incomplete spinal-cord injury. The article demonstrates how to set up the equipment; how to place stimulating and recording electrodes; and how to use the free EPOCS software to optimize electrode placement, measure the recruitment curve of direct motor and reflex responses, measure the response without operantly conditioning, condition the reflex, and analyze the resulting data. It illustrates how the reflex changes over multiple sessions, and how walking improves. It also discusses how the system can be applied to other kinds of evoked responses to other kinds of stimulation (e.g., motor evoked potentials to transcranial magnetic stimulation), how it can address various clinical problems, and how it can support research studies of sensorimotor function in health and disease.

stimulation), how it can address various clinical problems, and how it can support research studies of sensorimotor function in health and disease.

SUMMARY:
The Evoked Potential Operant Conditioning System (EPOCS) aids scientific investigation of sensorimotor function, and can administer targeted neurobehavioral training that can impact sensorimotor rehabilitation in neuromuscular disorders. This article describes its capabilities, and illustrates its application in modifying a simple spinal reflex to achieve lasting improvement in motor function.

INTRODUCTION:
Over the past decade, targeted neuroplasticity strategies have emerged as a new approach to rehabilitation of neurological impairments. 1,2 One such strategy is operant conditioning of an evoked potential. This entails repeatedly eliciting electrophysiological responses that can be measured non-invasively-for example by electroencephalography (EEG) or surface electromyography (EMG)-and giving the person immediate feedback on the size of each response relative to a criterion level set by the therapist or investigator. Over time, this protocol trains the person to increase or decrease their response, and can consequently target beneficial change to a central-nervous-system site that is important in a behavior such as locomotion or reach-and-grasp. The targeted change benefits performance and, in addition, enables better practice that leads to widespread beneficial change that improves the entire behavior. For example, in people with incomplete spinal-cord injury (iSCI) in whom clonus impairs locomotion, operant conditioning that reduces the Hoffmann reflex in the soleus muscle of one leg improves locomotor muscle activity in both legs, and thereby lastingly increases walking speed and restores right/left step symmetry. 1,[3][4][5] Another example is that paired-pulse stimulation can lastingly increase the size of the motor-evoked potential (MEP) to transcranial magnetic stimulation, thereby improving reach-and-grasp function in people with chronic hand and arm impairment following iSCI. 6 Implementing such protocols demands special-purpose software that must perform multiple functions. Specifically, it must continuously acquire, process & save electrophysiological signals; it must continuously monitor the state of the nervous system and trigger stimulation appropriately under tight real-time constraints; it must provide continuous feedback, trialby-trial feedback and session-by-session feedback; it must provide a user interface to guide setup and tuning by the investigator or therapist; finally, it must store and organize signal data and meta-information in a standardized format.
The Evoked Potential Operant Conditioning System (EPOCS) is our answer to this outstanding need. Under the hood, the EPOCS software is based on our open-source neurotechnology platform BCI2000, 7,8 which is used in hundreds of laboratories around the world. In EPOCS, the standard BCI2000 user interface is hidden, and replaced by a streamlined interface that is optimized for evoked-potential operant-conditioning protocols.
The current article and its accompanying video illustrate the use of EPOCS in one particular protocol: operant conditioning to reduce the size of the Hoffmann (or "H-") reflex in the soleus muscle. This response is the electrically-elicited analog of the "knee-jerk" stretch reflex. H-reflex down-conditioning has been shown to reduce the impact of clonus on, and to thereby improve, locomotion in animals with iSCI 9-13 and humans with iSCI, multiple sclerosis or stroke. 14,15,5 It can be applied without adverse side-effects in animals and people with or without neurological injury. 16,17 The operant conditioning protocol functions by performing multiple trials, each lasting several seconds. The sequence of events of one trial is shown schematically in Figure 1, with numbers denoting the following functions:

1.
Background (i.e., ongoing) EMG is recorded from bipolar surface electrodes over the target muscle (soleus) and its antagonist (tibialis anterior). Background level is evaluated as the mean rectified value of the high-pass-filtered signal in a sliding window.

2.
Background EMG level in the target muscle is shown as the height of a bar, continuously updated on the participant's screen. This helps the participant to keep the activity within a specified range (hatched region).

3.
The software judges the appropriate moment for electrical stimulation, and triggers the stimulator accordingly. The principal criteria are: at least 5 seconds must have elapsed since the previous stimulation; and background EMG level must have remained in the specified range continuously for 2 seconds.

4.
A constant-current stimulator delivers an electrical pulse transcutaneously to the tibial nerve (typically monophasic, with 1 millisecond duration).

5.
The resulting stimulus-locked response is recorded. The software computes the sizes of two components of particular interest: the earlier M-wave which reflects muscle activation resulting from direct stimulation of the motor axon, and the later H-reflex which reflects the signal relayed through a reflex arc in the spinal cord. [18][19][20][21][22] EPOCS refers to these as the reference response and target response, respectively.

6.
H-reflex size for the current trial is displayed as the height of a second bar, relative to a desired criterion level that defines a successful or unsuccessful trial. For down-conditioning, the bar is dark green if the H-reflex size fell below the criterion, or bright red if it did not (vice versa for up-conditioning). Simultaneously, the numeric display of the cumulative success rate is updated accordingly. Together, these graphical display elements provide the immediate positive or negative reinforcement on which operant conditioning relies. 23 To support this protocol as well as other related protocols, EPOCS provides five distinct modes of operation, each served by one of the tabs of its main window, entitled Stimulus Test, Voluntary Contraction, Recruitment Curve, Control Trials and Training Trials.
In Stimulus Test mode, the software triggers a stimulus every few seconds, not necessarily contingent on the state of the target muscle. The response signals are shown on screen after each stimulus. This allows the operator to verify the quality of the electrode connections and the EMG signal; to optimize the position of the stimulating and recording electrodes; and to establish the individual's response morphology.
In Voluntary Contraction mode, the software measures and shows background EMG level while the participant is encouraged to contract the muscle as much as possible, in the absence of electrical stimulation. In some protocols, the EMG level at maximum voluntary contraction (MVC) is a useful reference for setting background EMG criteria. In the particular protocol demonstrated here, this will not be necessary, as a stable standing posture standardizes the activity of the soleus muscle sufficiently.
In Recruitment Curve mode, stimulation is contingent on background EMG level (shown continuously on screen) remaining in the correct range; response signals are shown on screen after each stimulus; and the sequence of responses may be analyzed at the end of a run. This allows the operator to determine the start and end of the time intervals in which the responses of interest appear; to determine the relationship between stimulation intensity and response size, both before and after conditioning runs; and to determine the stimulation intensity to be used for conditioning.
In Control Trials mode, stimulation is contingent on background EMG level (shown continuously on screen), but no feedback is given about the target response size. The sequence and distribution of response sizes may be analyzed. This mode may be used to gather baseline measurements of response size, or as a control condition for comparison against operant conditioning in a crossover or between-subject experimental design. It can serve as a basis for setting the performance criterion for operant conditioning at the beginning of each session.
Finally, in Training Trials mode, stimulation is contingent on background EMG level (shown continuously on screen), and trial-by-trial reinforcement is also provided by showing the target response size, as described above and shown in Figure 1. This is the mode in which operant conditioning is performed.
The next section will guide the reader through the five modes by demonstrating the protocol for down-conditioning the soleus H-reflex in an adult participant without neurological injury.
1. Install and configure the software. This step only has to be performed once for a given hardware configuration. 3. Attach stimulation and recording electrodes in their starting positions.
3.1. Use any previously-noted landmarks or measurements to recreate previous participantspecific electrode positions as closely as possible.
3.2. Prepare the skin where electrodes will be attached by wiping with alcohol pads, to remove excess oil, then wipe with a paper towel to remove dead skin. 3.4. Attach EMG recording electrodes in a bipolar montage at the target muscle (soleus) as follows.
3.4.1. To determine the correct location, first find the gastrocnemius muscle by palpating while the participant alternates betweening standing on their toes and standing naturally.

3.4.2.
Place the first electrode directly below the distal border of the gastrocnemius muscle belly.
3.4.3. Place the second electrode below the first, with a 5 cm spacing between electrode centers. Keep both electrodes in line with the Achilles tendon.
3.5. Attach EMG recording electrodes in a bipolar montage at the antagonist muscle (tibialis anterior).  4.5.11.2. Use the mouse to adjust the beginning and end of the brown "reference" and green "target" intervals (in the H-reflex operant conditioning protocol, these correspond to M-wave and H-reflex, respectively). 4.5.11.3. When the intervals are correct, press the red Use Marked Timings buttons to save these personalized interval settings for future analyses.
4.5.12. In the Sequence pane in the lower half of the analysis window, assess the resulting recruitment curve. Adjust the settings to view either peak-to-peak or mean-rectified amplitude, and to pool results from consecutive trials (since the stimulus current was increased every four trials, specify Trials to Pool: 4

4.7.3.
If the participant has not seen it before, draw their attention to the new feedback bar in the middle of the screen. Explain that it shows the most recent H-reflex size relative to the hatched target range. If the response falls within the target range, the trial will be counted as successful and the bar will be dark green. If it falls outside the range, the trial will be counted as unsuccessful and the bar will be a brighter red.

Author Manuscript
Author Manuscript

Author Manuscript
Author Manuscript 4.9.1. Type up any additional session notes in the Log tab. The log is saved automatically as it is filled in, in a date-stamped plain-text file in the session-specific data directory.
4.9.2. Close the EPOCS window. Data and logs will already have been saved.
4.9.3. To revisit the Analysis window for previously-recorded data, double-click the EPOCS Offline Analysis icon, and select the data file for the run to be analyzed. Wait for the raw signals to be processed (this may take a minute or more). NOTE: Data are saved as .dat files in BCI2000 format. The file name indicates the date and time of the session, the participant ID, the mode (ST for Stimulus Test, VC for Voluntary Contraction, RC for Recruitment  Curve, CT for Control Trials and TT for Training Trials), and the sequential run number.
5. Repeat the session. the beneficial effect of soleus H-reflex down-conditioning in participants with chronic lower limb impairment following incomplete spinal cord injury. The bars contrast results for N=6 participants whose H-reflex was successfully down-conditioned, against N=4 participants from the control condition (no operant conditioning) and N=3 participants in whom the down-conditioning protocol failed to reduce reflex size. Successful conditioning was associated with an improvement in gait symmetry, and in walking speed relative to baseline-see  for further details. 14 DISCUSSION: The protocol described above is suitable for demonstrating soleus H-reflex downconditioning in a typical adult without neurological impairment. The precise parameter values may vary from person to person, and particularly as a function of impairment. Whereas the participant's recruitment curve reached M max at a stimulating current of around 25 mA in the video, another person might require 50 mA or more, so the current would be increased in larger steps during recruitment curve measurement. A longer pulse duration may also be required. A third person might be more sensitive, and require smaller current settings. The protocol will also need to be adapted according to the muscle that is being conditioned. For example, when targeting the flexor carpi radialis muscle 24,25 : a lower current setting is generally used; the Voluntary Contraction mode should be used to establish a scale for the background-EMG limits; and greater care must be taken both during optimization of electrode placement, and during optimization of posture, which must then be kept constant across trials.
The protocol is sensitive to variations in the relationship between stimulator current setting and the amount of current actually delivered to the nerve-this may be affected by small variations in posture, hydration of the participant, and drying out of the adhesive electrode gel. In H-reflex conditioning, this problem can be mitigated by using M-wave size as an indicator of effective stimulation intensity. It reflects the number of soleus motoneuron efferent axons excited by the stimulus. Thus, if M-wave size is kept constant, it implies that the number of primary afferent axons excited by the stimulus (i.e., the axons that elicit the H-reflex) is also kept constant (see also Crone et al., 1999 26 ). Hence, this M-wave is referred to as the "reference" response in the software. For this reason, step 4.5.14 mentions that the target M-wave size should be recorded: it is actually more important to keep this response size roughly constant than to keep the nominal current strictly constant. The Sequence tab of the EPOCS Analysis window allows retrospective verification of M-wave constancy over each run; for soleus H-reflex conditioning, this is often sufficient to correct any problems. For greater control, a second monitor may be attached to the computer, where EPOCS can provide real-time M-wave analytics to inform trial-by-trial manual adjustment. Automation of this control task is an ongoing project. 27 The success of operant conditioning may be sensitive to the accuracy of the time interval chosen by the operator to define the H-reflex; in particular, the interval should not be too wide. Detailed guidelines for correct interval definition are beyond the scope of the current article. This is also a function that will be automated in future versions of the software.
A critical step in the protocol is 4.5.7, in which the operator manually increases the stimulator current repeatedly after each fixed number of trials. Mis-counting the trials here, or mis-adjusting the current dial, can lead to distortion of the resulting recruitment curve. This possibility of user error can be mitigated by enabling the Digitimer Link option, which allows automation of the current adjustment for one particular stimulator model.
This article has focused on H-reflex conditioning, as it is the most fully developed of the potential clinical applications of EPOCS. The existing software helps researchers in the ongoing efforts to hone this particular protocol toward wide clinical dissemination. 32 Beyond H-reflex conditioning, EPOCS may also be applied in its current form to a wider variety of stimulation methods and evoked responses. For example, it can equally well trigger a mechanical device that elicits a stretch reflex, which may also be conditioned. [33][34][35] The approach is adaptable to an individual's impairments: in one person, down-conditioning the soleus H-reflex improves locomotion by reducing spastic hyperreflexia 14 ; in another, up-conditioning the tibialis anterior MEP improves locomotion by alleviating foot drop. 36 While efforts are ongoing to produce a commercial implementation of the protocol, the original EPOCS software will be maintained in parallel as a research tool, to provide the necessary flexibility to expand the field of targeted neuroplasticity. This flexibility is enabled by the modularity and extensibility of the widespread and well-established BCI2000 software platform, on which EPOCS is based. This means that, with minimal intervention by a software engineer familiar with BCI2000, the system is re-configurable for an even wider variety of research purposes. For example, it can be configured to record additional biosignal channels or additional sensors for later analysis (e.g. foot switches and motion tracking sensors, for conditioning during locomotion). It can also be programmed to consider additional triggering criteria for stimulation (e.g., triggering stimulation only at a particular part of the gait cycle); or to trigger additional reinforcement stimuli on successful or unsuccessful trials. Example customization files are provided, using BCI2000's scripting syntax.
Targeted neuroplasticity is still in its infancy. Its as-yet unexplored avenues are expected to provide great benefits both for developing novel therapeutic approaches (as discussed above), and for elucidating the natural history of disease, and mechanisms of central nervous system function in both health and disease. 2,32,37 We are therefore committed to maintaining and supporting EPOCS as a key tool for realizing this therapeutic and scientific potential. The participant views a large monitor screen that shows the background EMG level, the most recent H-reflex size, the number of trials completed so far in the current run of 75, and the running proportion of successful trials for the run. The sequence of events in one trial is denoted by the numbers 1-6 as described in the Introduction.